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ABSTRACT 

This paper, in a fashion easy to follow, illustrates 
the interesting relationship between structural equation modeling and 
canonical correlation analysis. Although computationally somewhat 
inconvenient, representing canonical correlation as a structural 
equation model may provide some information which is not available 
from conventional canonical correlation analysis. Hierarchically, 
with regard to the degree of generality of the techniques, it is 
suggested that structural equation modeling stands to be a more 
general approach. For researchers interested in these techniques, 
understanding the interrelationship among them is meaningful, since 
our judicious choice of these analytic techniques for our research 
depends on such understanding. Three tables and two figures present 
the data. (Contains 25 references.) (Author) 
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Abstract 

The paper, in a fashion easy to follow, illustrates the 
interesting relationship between structural equation modeling and 
canonical correlation analysis. Although computationally 
somewhat inconvenient, representing canonical correlation 
analysis as a structural equation model may provide some 
information which is not available from conventional canonical 
correlation analysis. Hierarchically, with regard to the degree 
of generality of the techniques, it is suggested that structural 
equation modeling stands to be a more general approach. For 
researchers interested in these techniques, understanding the 
interrelationship among them is meaningful, since our judicious 
choice of these analytic techniques for our research depends on 
such understanding. 



ERLC 



3 



SEM Se Canonical Analysis 1 
INTRODUCTION 

The utilization of multivariate methods has been widely 
recognized as to be important in social and behavioral science 
research. The importance mainly stems from two considerations: 
1) we intend to honor the complex social reality in which we 
operate and which we eventually want to generalize to; 2) we 
intend to avoid inflating experiment-wise error rate in our 
statistical analysis (Fish, 1988; Johnson & Wichern, 1988; 
SAS/STAT User's Guide, Version 6, Vol. 4, 1989; Stevens, 1986). 
Among the multivariate statistical techniques, canonical 
correlation analysis has occupied an important strategic 
position. It has often been conceptualized as a unified approach 
to many parametric statistical testing procedures, univariate and 
multivariate alike (Baggaley, 1981; Dunteman, 1984; Fornell, 
1978; Knapp, 1978; Kshirsagar, 1972; SAS/STAT User's Guide, 
Version 6, Vol. 4, 1989; Thompson, 1984, 1991). 

Many popular parametric statistical techniques share one 
thing in common: they are designed to analyze linear 
relationships among variables. It has been shown that many of 
the seemingly different techniques are really not that different 
from one another after all (Fan, 1992; Knapp, 1978; Thompson, 
1991) . More specifically, "most of the practical problems 
arising in statistics can be translated, in some form or the 
other, as the problem of measurement of association between two 
vector variates S and y" (Kshirsagar, 1972, p. 281). Canonical 
correlation analysis, which summarizes the relationships between 
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SEM & Canonical Analysis 2 

two groups (vectors) of variables, naturally brings order to the 
superficial chaos of a myriad of analytic techniques. 
Theoretically or empirically, the interesting relationship 
between canonical correlation analysis and many other statistical 
testing procedures ranging from simple correlation, t~test, to 
MANOVA, discriminant analysis has been shown many times (Knapp, 
1971; Kshirsagar, 1972; Tatsuoka, 1989; Thompson, 1991). 

Structural equation modeling (SEM) has been heralded as a 
unified model which joins methods from econometrics, 
psychometrics , sociometrics, and multivariate statistics 
(Bentler, 1994) . The generality and wide applicability of 
structural equation modeling approach has been amply demonstrated 
(Joreskog & Sorbom, 1989; Bentler, 1992). The statistical 
relationship among structural equation modeling and canonical 
correlation analysis, however, is less obvious. Consequently 
such relationship is less known by researchers, despite the 
insightful discussion on this topic provided by Bagozzi, Fornell 
and Larcker (1981). The purpose of this paper is to use concrete 
analysis examples to show the relationship among structural 
equation modeling and canonical correlation analysis. Also, this 
paper attempts to raise some questions about whether the 
structural equation modeling approach to canonical analysis will 
allow us to accomplish more than conventional canonical 
correlation approach. For researchers who are interested in 
these sophisticated analytical tools, to understand the 
underlying relationship among these seemingly different methods 
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SEM & Canonical Analysis 3 

will contribute to their judicious choice among these techniques 
as they encounter different research situations. 

SEM AND CANONICAL CORRELATION ANALYSIS 
Canonical Correlation Analysis 
Pioneered by Hotelling (1935), canonical correlation 
analysis seeks to identify and measure the association between 
two sets of variables. Canonical correlation analysis can be 
understood as the bivariate correlation of two synthetic 
variables which are the linear combinations of the two sets of 
original variables (Johnson & Wichern, 1988; Thojnpson, 1984, 
1991) . 

The two sets of original variables are linearly combined to 
produce pairs of synthetic variables which have maximum 
correlation, with the restriction that each member of each 
subsequent set of such synthetic variables is orthogonal to all 
members of all other sets. The maximum number of such pairs of 
synthetic variables which can be produced equals the number of 
variables in the smaller set of the two. In this sense, the 
synthetic variables in canonical correlation analysis, which are 
the linear combinations of the original variables, are similar to 
the synthetic variables produced in some other multivariate 
analysis techniques suchr as principal component analysis, 
discriminant analysis, etc. The difference is that, in different 
statistical analysis, the original variables are linearly 
combined to satisfy different criteria. For example, in 
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SEM & Canonical Analysis 4 

discriminant analysis, the original variables are linearly 
combined to produce synthetic variables which maximizes the ratio 
of between-group variance to within-group variance, so that 
different groups can be maximally differentiated on the synthetic 
variables. In principal component analysis, the synthetic 
variables (principal components) are constructed so that the 
variance on the synthetic variables are maximized, and the 
maximum amount of variance in the original variables can be 
accounted for by the smallest number of these principal 
components • 

As can be expected for a multivariate statistical method, in 
canonical correlation analysis, eigenstructures of certain 
matrices play a crucial role in deriving the linear coefficients 
needed to produce the synthetic variables and in deriving the 
canonical correlation coefficients for different canonical 
functions. Assume two sets of variables X and Y, with X set is 
the smaller of the two. When combined, the two sets of variables 
have the following partitioned correlation matrix: 



R = 



The derivation of the linear coefficients for combining 
original variables into canonical variates is based on the 
following two matrices A and B, which have the same eigenvalues 
Xi, but have different eigenvectors and bi associated with the 
eigenvalues . 
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SEM & Canonical Analysis 5 



^ = ^xy ^ ^yx 

The elements in the eigenvectors ai and bi turn out to be 
the linear coefficients needed for the two original sets of 
variables X and Y respectively. These coefficients are often 
referred to as canonical function coefficients (Thompson, 1984) 
or canonical weights (Pedhazur, 1982) • Once the linear 
coefficients are obtained, we use them to derive a pair of 
synthetic variables (canonical variates) : 

The correlation between X*i and Y*i is maximized, subject to the 
restriction that each subsequent canonical function is orthogonal 
to all previous canonical functions. The square root of the 
associated eigenvalue ki is the correlation between X* and Y* 
(Rx*y*= ^i^) • Once the first eigenvectors and associated 
eigenvalue are determined, a second pair of canonical variates 
can be derived based on the next eigenvector-eigenvalue pair* 
Canonical structural coefficients, i.e., the correlation 
coefficients between the original variables (X and Y) with their 
respective canonical variates, can be obtained once canonical 
function coefficients are known by premultiplying the function 
coefficient vectors with the associated correlation matrix of the 
original variables: 
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For every observed variable, it is not only correlated with 
its own canonical variate of its own set, but it may also be 
correlated with the canonical variate of the other variable set. 
This correlation is termed "cross-loading" (Bagozzi, Fornell, & 
Larcker, 1981) or "index coefficients" (Johnson & Wichern, 1988; 
Thompson, 1984) . The relationship between index coefficients and 
function coefficients are as follows: 

^x*y ^ 

Furthermore, the index coefficients are related to canonical 
correlation coefficient (rc)as follows: 



y 

= / I?"* r 

y* X XX 'y* X 



structural Equation Modeling 
Structural equation models have been discussed in literature 
extensively (J-reskog & S-rbom, 1989; Bentler & Weeks, 1980). 
Different theoretical models have been proposed which essentially 
accomplish the same goals (McDonald, 1978, 1980; McArdle, 1980; 
Bentler & Weeks, 1980; J-reskog & S-rbom, 1988). Among these 
models, EQS (Bentler, 1992; Bentler & Weeks, 1980) and LISREL 
(J-reskog & S-rbom, 1988) models have become widely known among 
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researchers in different substantive research areas ♦ 

EQS model represents the linear relationship among variables 
(either observed or latent) as: 

tl = f^Tl + Y$ 

Where, correspond to endogenous variables (either observed or 
latent), while 5 correspond to exogenous variable (also either 
observed or latent) . y describes the relationship from exogenous 
to endogenous variables, and p describes the relationship between 
endogenous to other endogenous variables ♦ 

Similar to EQS model but with somewhat more elaboration, 
LISREL model specifically distinguishes between relationships 
between observed and latent variables (two measurement models for 
exogenous and endogenous latent variables respectively) and those 
between latent variables themselves (structural model) : 

» 

structural Model and Canonical Analysis 
SEM Representation of Canonical Correlation Analysis 

To represent canonical correlation analysis using structural 
equation modeling turns out to be not so straight forward • 
Following the demonstrations by Bagozzi et al. (1981), the 
canonical variate pair (X* and Y*) can be reduced to a model with 
a single latent variable (either X* or Y*) , rather than two 
latent variable model (both X* and Y*) • Based on this approach, 
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the canonical correlation analysis can be represented as a 
Multiple Indicators/Multiple Causes (MIMIC) model. The 
distinguishing feature of MIMIC model is that the latent variable 
has both causal indicators and effect indicators (MacCallum & 
Browne^ 1993) • The application and implementation of MIMIC model 
in research has been discussed elsewhere (Goldberger, 1972; 
Joreskog & Sorbora, 1988) . The MIMIC model representing the first 
canonical variate X*i is presented in Figure 1. 

Insert Figure 1 about here 



In the model of Figure 1, the F matrix (the effects of 
causal indicators on the latent variable r\) contains function 
coefficients for the first canonical function for X variables. 
The Ay matrix (the effects of the latent variable t| on its effect 
indicators) contains the index coefficients of Y set variables on 
the first canonical variate X*i. As discussed before, this SEM 
representation of canonical correlation analysis only deals one 
canonical variate (X^* or Y^*) at a time, and in this case, the 
latent variable y\ is actually the first canonical variate X*i for 
X variables. 

Since the latent variable r\ in Figure 1 represents the first 
canonical variate X*^, haseC. on the concept of canonical 
correlation that the canonical variate is a linear combination of 
observed variables without error, the disturbance term (C) for 
the latent variable x\ should be fixed to be zero. Also, since 
canonical correlation analysis is symmetrical, canonical variate 
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Y*i for Y variable set can easily be represented by reversing the 
direction of the MIMIC model, i*e*, by changing the causal 
indicators into effect indicators, and the effect indicators into 
causal indicators • 

For the first canonical function, once canonical function 
coefficients and index coefficients are obtained from SEM model, 
canonical correlation coefficient and canonical structure 
coefficients can be derived based on the formulas presented 
before. After the first canonical function depicted in Figure 1 
is solved, the second canonical function can be built over the 
first by imposing another latent variable on the- MIMIC model, and 
constraining all the coefficients of the original model to be 
equal to the values already obtained for the model. Since 
canonical functions are orthogonal to each other, the new latent 
variable is specified to have or receive no effect from the first 
latent variable* The first two canonical functions are depicted 
in Figure 2. In the same vein, more latent canonical variates 
can be sequentially added to the MIMIC model to accommodate more 
canonical functions as data permit. 

Insert Figure 2 about here 



From the discussion above, it appears somewhat cumbersome to 
represent canonical correlation analysis using SEM approach, 
since several related models have to be analyzed and some 
additional calculation is also necessary to obtain all the 
results typically obtained in a canonical correlation study. 

o 12 
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SEM & Canonical Analysis 10 

These inconveniences aside, SEM approach does seem to be able to 
provide some additional information which canonical correlation 
analysis does not have. Two such potential advantages are 1) 
significance testing for canonical function coefficients; 2) 
significance testing for individual canonical functions. 
Significance Testing for Coefficients 

For some multivariate methods such as discriminant analysis, 
factor analysis, etc, , theoretical sampling distributions are 
either not available or extremely complex for different types of 
coefficients, thus it is difficult or impossible to conduct 
statistical significance testing for these coefficients. The 
same is true for canonical correlation analysis. 

The lack of sampling distributions for many coefficients in 
some multivariate methods contributes to the subjectivity of 
certain some decision process. For example, in canonical 
correlation analysis, what value of function coefficients should 
we use as the criterion to judge the importance of contribution 
of observed variables to the canonical function? Since 
significance testing is not available in canonical analysis for 
the coefficients, we have one less tool for the purpose, SEM 
approach to canonical analysis seems to be capable of providing 
us with this information. In SEM approach depicted in Figure 1, 
since F and Ay contain canonical function and index coefficients 
respectively, and standard errors for these coefficients are also 
available from SEM, critical t ratios can be calculated to give 
us some sense about how much sampling error we can expect for 
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these coefficients. 

Testing for Individual Canonical Functions 

Mathematically, the likelihood ratio test in canonical 
analysis compares the sample restricted generalized variance 
under the null hypothesis with the unrestricted generalized 
variance as follows; 





0 






0 










^xy 




^xx ^xy 




Ryy 




^yx ^yy 



This test indicates that the necessary and sufficient condition 
for canonical correlations to be zero is ^ (Ryx) being zero. 

Due to the complexity of distribution theory for sample 
canonical correlation coefficients (Johnson & Wichern, 1988; 
Kirsagar, 1972) , the likelihood ratio test in canonical analysis 
is constructed not for testing individual canonical functions, 
but rather, for sequentially testing several canonical functions 
as a group. For example, in a situation where three canonical 
functions are derived, three likelihood ratio tests will be 
conducted. The first tests the Ho that all canonical functions 
are zeros, the second tests the Ho that the second and the third 
canonical functions are zeros, and the third tests the Ho that 
the last canonical function is zero. If the first two tests are 
statistically significant and the third is not, it will be 
concluded that the first two canonical functions are 



14 



SEM & Canonical Analysis 12 

statistically significant while the third is not. Such 
conclusion sounds as if we had conducted significance tests for 
individual canonical functions when, in reality, we have riot. 
Strictly speaking, only the last test for the third canonical 
function is a true test for individual function, and all previous 
tests are not. Theoretically, in the situation described above, 
it is possible that the second canonical function by itself ^oes 
not account for a statistically significant portion of data 
covariance, but only together with the third canonical function, 
they jointly account for a statistically significant portion of 
data covariance. This possibility will not be known under the 
conventional likelihood ratio test in canonical analysis. 

If structural equation model can be formulated to represent 
canonical analysis, one interesting question will be whether 
statistical significance test can be conducted for individual 
canonical function, instead of the sequential testing process 
described above. In other words, can we independently test for 
the null hypothesis that the first canonical correlation itself 
is zero, or the second, and the third, instead of testing for all 
three, last two, and last one? Conceptually, this seems to be 
possible in SEM using the approach of nested models and then test 
for the statistical significance of differential between two 
nested models, one with constraints while the other without. 

Within the framework of LISREL model, the implied 
correlation matrix for standardized variables for our MIMIC model 
is as follows: 
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Aj\>A', A^i^r' a; 

A^r^Ai (r<|)r^ a; + e. 

It is easy to see that once Ay is constrained to be zero, R^y (Ryx) 
will be zero so that the restricted condition for the likelihood 
ratio test in conventional canonical analysis will be satisfied. 
This suggests that constraining the Ay to be zero conceptually 
represents a restricted model of zero canonical correlation. 
That zero Ay means zero canonical correlation is also shown 
through the relationship between canonical index coefficients 
(r^* y = Ay) and canonical correlation coefficient, (r^,) : 

~ ^x* y ^x* y 

= /?"^ r 

'y* X .\x ' y* X 

The flexibility of testing for nested models approach in SEM can 
easily accommodate testing for such unrestricted vs restricted 
models, though in this case of MIMIC model, the restricted model 
seems to be a somewhat strange model with only causal indicators 
and no effect indicators. It is not clear what methodological 
ramifications such a restricted model may have on SEM estimation 
process. 

An Example 

An example in SAS STAT User's Guide (SAS Institute, 1988) is 
used as an illustration for the relationship between canonical 
correlation analysis and SEM. The data contain three physical 



R = 



^xx ^xy 
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variables (measurements of Weight, Waist and Pulse) and three 
exercise variables (numbers of Chins, Situps and Jumps 
completed) • The correlation matrix for the three variables is 
presented in Table 1, and the sample size was altered in order to 
yield statistically significant results for the canonical 
functions derived from the data (original sample size for the 
data matrix is 20) . 

Table 2 presents the results of canonical function 
coefficients from both SEM and canonical analysis approach* The 
values for the coefficients from the two approaches are 
essentially the same, thus not repeated in the table. The 
critical t ratios for the function coefficients based on SEM 
approach {t ratios are for unstandardized coefficients) are also 
presented in the table together with the coefficients. As 
discussed previously, conventional canonical analysis does not 
provide estimate for standard errors for the coefficients, thus 
making it difficult to make statistical assessment for the 
relative importance of the function coefficients, i.e., the 
relative contribution of observed variables to the canonical 
variate. In SEM approach, such information becomes available, 
assuming that the data satisfy the SEM theoretical assumptions in 
order for such estimates to be valid. 

It is observed that statistical importance of the 
coefficients may not be accurately assessed by its absolute 
value, since such values should be assessed in relation to its 
standard error. For example, the variable Chins has function 
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coefficient -.35 on the first canonical function, and -.37 on the 
second canonical function. Without knowledge of the standard 
errors, as in conventional canonical analysis, it may be 
concluded that the variable Chins is about equally important for 
the first and second canonical functions. But when judged in 
relation to the standard errors, the interpretation will change 
dramatically. This variable is much more important for the first 
canonical function (critical ratio 4.59) than for the second 
canonical function (critical ratio .88, and statistically not 
different from zero) . This example shows some advantage of using 
SEM to represent conventional canonical correlation analysis. 
Such information about standard errors for the coefficients is 
meaningful and important in the statistical sense. 

Table 3 presents the results of SEM nested-model approach 
for testing each of the three individual canonical functions as 
discussed above, together with the results from the sequential 
testing in canonical correlation analysis. It is seen that for 
this dataset, the results from the two approaches are very 
similar. It is unclear whether this (the same results based on- 
the two approaches) will hold for different data sets. 
Theoretically, the sequential testing approach in canonical 
analysis is somewhat different from testing individual canonical 
functions, since more than one canonical functions are being 
tested at a same time except for the last canonical function. 
Assuming that the nested model approach in SEM for testing 
canonical correlation as discussed above is correct, there seems 
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to be some slight conceptual difference between this and the 
sequential testing approach. It is unclear why no obvious 
difference between the two approaches occurs. 

The solution to the model presented in Figure 1 does not 
directly provide canonical correlation coefficient or canonical 
structural coefficients. By using formulas presented previously, 
these can be obtained with some calculation. Although this may 
be burdensome computationally, it does not negate the fact that 
canonical correlation analysis can be conceptually and 
statistically represented as a structural equation model. Such a 
relationship is meaningful in the sense that it further 
demonstrates the generality and versatility of structural 
equation modeling approach to data analysis. 

At this time, several questions remain with regard to the 
relationship between canonical analysis and structural equation 
modeling. First, is there a better model representation for 
canonical analysis other than what was presented in Bagozzi et 
al. (1981)? can we represent canonical analysis using a 
different model so that we may directly obtain those statistics, 
especially canonical correlation coefficient, which we can not 
directly obtain from the current model? Several models have been 
tried, but this current model seems to be the only one which 
works . 

Another question is related to the nested model testing 
approach for testing individual canonical functions. As 
discussed previously, it is not certain the approach tried in 
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this paper is a sound approach statistically, since the 
restricted model seems to be a strange model, although the test 
based on the differential seem to provide reasonable results. 
Are there better ways to construct statistical test for 
individual canonical function? Also on this point, the 
conventional sequential testing approach and the nested model 
testing approach seem to provide very similar results. Assuming 
that the nested model testing approach is correct, is this a 
coincidence due to data, or they are essentially the same? These 
and some other minor questions aside, the relationship between 
conventional canonical analysis and structural equation modeling 
is unmistakable. Researchers interested in these methods may 
benefit from the understanding of such underlying relationship. 
The statistical relationship between the approaches may also 
provide us with some new insight for the conventional canonical 
correlation analysis. 
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Table 1 

Correlation Data Matrix Used for Presentation (N = 213) 



Weight 


1.000 










Waist 


0.870 


1.000 








Pulse 


-0.366 


-0.353 


1.000 






Chins 


-0.390 


-0.552 


0.151 


1.000 




Situps 


-0.493 


-0.646 


0.225 


0.696 


1.000 


Jumps 


-0.226 


-0.192 


0.035 


0.496 


0.669 
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Table 2 

Testing for Canonical Functions; SEM Nested Model Approach and 
Sequential Testing Approach in Canonical Analysis 



SEM Sequential Testing 



Ho: 


1st 


rc=0 


X^dif f (df-3) ^ 


212.01 


Ho: 


All 3 rc=0 








p < .0001 






p < .0001 


Ho: 


2nd 


rc=0 


X^diff 


8.62 


Ho: 


Both 2nd & 3rd rc==0 








p < .05 






p < .05 


Ho: 


3rd 


rc=0 


X^diff <df-3)^ 


1.12 


Ho: 


The 3rd rc==0 








p > .10 






p > ;10 
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Table 3 

Standardized Canonical Function Coefficients and Critical Ratios 
from Canonical Analysis and SEM Estimation 

Variable Variable Function Function Function 



Group 


Names 


I 




II 






III 


Physical 


Weight 


-.77 


(7.23)* 


-1.88 


(11.44) 


-.20 


(.11*) 




Waist 


1.57 


(12.91) 


1. 17 


(4.07) 


.51 


(.35*) 




Pll1 c;p 




(1 04*^ 
^ J. . yjH ) 




( . 64*^ 


1 . 05 


( 1 . 04*^ 

^ J- . ; 


Exercise 


Chins 


-.35 


(4.59) 


-.37 


(.88*) 


-1.30 


(1.59*) 




Situps 


-1.05 


(11.25) 


.12 


(.21*) 


1.24 


(2.07) 




Jumps 


.72 


(9.60) 


1.06 


(5.53) 


-.41 


(.37*) 


Canonical 


r 


rci = 


.795 


ro2 = 


.199 


rc3 = 


.072 


SEM Fit 




X^df«3~ 


9.73 


X^df-3~ 


1.12 


X^df»3~ 


.000 



p < .001 p > .10 

a critical ratios estimated from SEM in parenthesis 
* statistically not different from zero 
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Figure 1: Structural Equation Model Representation of the First 
Canonical Function 
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Figure 2 Structural Equation Model Representation of the First 
Two Canonical Functions 



ERIC 
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